This is a complete script for analyzing mouse trajectories collected in the mouse-tracking Social Simon task. The complete experiment consisted of 4 conditions:
The script goes from data-preprocessing through various ways to analyze mouse-tracking data that have been applied in the literature. This version of the script carries out analysis only on the condition 3 as it is of highest interest in this study. It can be applied directly to analyzing data from condition 4 and with slight modifications to conditions 1 and 2.
First of all we save the screen dimensions and relevant coordinates used in the experiment.
# save experiment parameter values
screen.width <- 1920
screen.height <- 1080
start.boundary <- 988 # upper boundary of the start button
response.boundary <- 128 # lower boundary of response boxes
stim.boundary <- 820 # the y-coord that needed to be crossed for stimulus to appear
We then begin by loading the data and reformatting it for easier processing.
Then we add the coding for independent variables on
The y-coordinates are immediately flipped vertically because the package that was used for collecting the data (Matlab Psychtoolbox) encodes the screen’s top left as coordinates [0, 0] and therefore y-coordinates grow towards the bottom of the screen while for ease of analysis we would like them to grow towards the screen’s top.
We plot an example trial before any further pre-processing. The complete trajectory that is plotted contains all coordinates recorded since the start of the trial and so also before the participants clicked on the start button located at the bottom of the screen.
Now we filter out only successful trials, in which participants did not miss any deadlines and a correct response was given.
Complete trajectories will be needed for the dynamical part of our analysis at the end of this script. For remaining analyses we need to extract the portions of the trajectories after participants have clicked on the start button.
For convenience we rescale the coordinates into a standard MouseTracker coordinate space, where x is in range [-1, 1] and y in range [0, 1.5].
Now we align all the trajectories to the common [0, 0] origin and timestamps to start at 0.
At this point we can already visualize all trajectories of all individual participants and pairs.
We see that most pairs seem to divide the screen space between each other by moving mostly directly towards their assigned response box and avoiding the center. However, there are exceptions.
In condition 3 (with visual feedback) one particular pair (number 7) has mostly upward moving trajectories. We can further explore here whether the joint upward motion is induced by one of the participants or happens immediately on both sides.
The plot presents trajectories in the first couple of trials (2, 3, 5, 6, 7) of two participants from pair 7. The trajectories are colored red for when participant’s role was “active”, i.e. the cue that appeared had their assigned color and blue for when their role was “passive”, i.e. their task in such a trial was to not press the response button and simply go back to the start position.
From the plots it would appear that one of the members of this aberrant couple (person 2) adopts a “move upward” strategy from the start, independently of whether it is their turn to respond or not. The other person in that couple seems to copy the co-actor’s movements on passive trials and make large discrete errors on the trials in which it is actually their turn to respond. Why this particular couple behaves in this manner is unfortunately unknown and might indicate some individual differences, the feeling of jointness experienced by this couple or conscious strategies that people adopt in such a task.
In condition 4 (without visual feedback) it is rather than certain individuals adopt the “move upward” strategy, independently of their partner (which is to be expected since they do not see the partner’s movements).
As the next pre-processing step we will flip all trajectories to one side. This ensures that every trajectory starts at the bottom of the coordinate system and ends in the top right corner. It is done to obtain comparable trajectory measures.
As a last step, a look at sampling rate distribution to see whether it reveals any outliers that could indicate missing data or wrong recording.
From the data we can extract all intervals between adjacent sampling points and calculate their mean and standard devation. We then establish a cutoff point of 3 SD from the mean beyond which the sampling interval is considered to be an outlier. We calculate the number of such outliers for each condition and their mean sampling rate. The resulting numbers are presented in the following table.
| rate.means | rate.sds | rate.cutoff | num.outliers | mean.outliers | |
|---|---|---|---|---|---|
| condition1 | 10.87 | 0.01 | 10.88 | 596 | 10.90 |
| condition2 | 10.87 | 0.01 | 10.91 | 8 | 13.60 |
| condition3 | 10.87 | 1.30 | 14.78 | 41 | 207.71 |
| condition4 | 10.87 | 0.53 | 12.47 | 39 | 42.92 |
It turns out that the means sampling rate for all conditions is within expected parameters given the set sampling rate of 92 Hz. The number of outliers and their means varies per condition with much larger deviations in social conditions. Given the cross-computer data stream in these conditions, some amount of data loss was to be expected. In order to facilitate binned data analysis, we remove trials that contain these large deviations (separately for each condition).
Next we look at y coordinates that should not be lower than some margin around 0 (after flipping and alignment). A y-coordinate that is more negative indicates a faulty recording of the start button press.
With the sampling timing issues fixed, we can examine reaction time outliers and add another variable to the data, which indicates whether the trial was fast or slow.
Note that in condition 2 the histogram looks markedly different for active and passive trials. This is because when participant was supposed to refrain from responding, they had to simply wait for the trial to end.
After we produced clean data for all conditions, we combine the two social ones into a single data frame.
For the analysis we will use a mousetrap package. Accordingly, the next step is to transform the data into a mousetrap object.
As a final pre-processing step, we perform time normalization on the data, in which the times and coordinates are linearly interpolated so that each trajectory contains the same number of recorded points, typically this is set to 101 points.
We need to check that all participants have a balanced number of observations for different variables of interest.
From the counts plot, we can see that in condition 1 and 2 one person in each has markedly less successful trials than other participants. In condition 3, even though different numbers of trials remain for different pairs after removing unsuccessful ones, there are similar counts for different trial and role types.
By contrast, in condition 4, three of the pairs completely lose observations for one of the incongruent set of trials, namely participants with ids 14, 23, 29 lose incongruent active trials while their co-actors (ids 13, 24, 30) incongruent passive ones. Further investigation of these counts reveals that the crucial step that leads to this loss is filtering out trials in which incorrect response was given, that is, participant clicked on the wrong response box. Given that this correlates with trials being incongruent (the cue appeared on the same side as the incorrect response box), we might infer that participants 14, 23 and 29 misunderstood the task, i.e. they were responding to the location of the cue, rather than its color. We can confirm this conclusion by plotting the trajectories of these participants.
We see that indeed, 3 participants in condition 4 misunderstood the instructions and therefore need to be removed from further analysis.
As a result of data cleaning, 13%, 12%, 4%, 27% of trials in conditions 1 to 4 are removed.
There is a number of measures that can be calculated on the basis of raw time and normalized trajectories. First, we retrieve trajectory derivatives (velocity, acceleration) and angles from the raw time data. Next, we calculate a variety of measures on normalized data.
These are the measures that are returned:
| Measure | Definition |
|---|---|
| xpos_max | Maximum x-position |
| xpos_min | Minimum x-position |
| ypos_max | Maximum y-position |
| ypos_min | Minimum y-position |
| MAD | Signed Maximum absolute deviation from the direct path connecting start and end point of the trajectory (straight line). If the MAD occurs above the direct path, this is denoted by a positive value; if it occurs below, by a negative value. |
| MAD_time | Time at which the maximum absolute deviation was reached first |
| MD_above | Maximum deviation above the direct path |
| MD_above_time | Time at which the maximum deviation above was reached first |
| MD_below | Maximum deviation below the direct path |
| MD_below_time | Time at which the maximum deviation below was reached first |
| AD | Average deviation from direct path |
| AUC | Area under curve, the geometric area between the actual trajectory and the direct path where areas below the direct path have been subtracted |
| xpos_flips | Number of directional changes along x-axis (exceeding the distance specified in flip_threshold) |
| ypos_flips | Number of directional changes along y-axis (exceeding the distance specified in flip_threshold) |
| xpos_reversals | Number of crossings of the y-axis |
| ypos_reversals | Number of crossings of the x-axis |
| RT | Response time, time at which tracking stopped |
| initiation_time | Time at which first mouse movement was initiated |
| idle_time | Total time without mouse movement across the entirety of the trial |
| hover_time | Total time of all periods without movement in a trial (whose duration exceeds the value specified in hover_threshold) |
| hovers | Number of periods without movement in a trial |
| total_dist | Total euclidean distance covered by the trajectory |
| vel_max | Maximum velocity |
| vel_max_time | Time at which maximum velocity occurred first |
| vel_min | Minimum velocity |
| vel_min_time | Time at which minimum velocity occurred first |
| acc_max | Maximum acceleration |
| acc_max_time | Time at which maximum acceleration occurred first |
| acc_min | Minimum acceleration |
| acc_min_time | Time at which minimum acceleration occurred first |
Out of these measures, the most widely used are measures of the curvature of mouse trajectories, i.e. MAD, AD and AUC. A variety of time-based measures allows us to examine movement duration in different stages of the trajectory. Finally, coordinate flips and reversals are a proxy for movement complexity (which we later examine also with entropy-based measure).
For the purpose of this paper we will focus on the shape of movement trajectories and velocity profiles in the exploratory part of our analysis. In the inferential part we will examine only overall reaction time RT, area under curve AUC and sample entropy ENT.
Our data for conditions 2 to 4 contains trials in which a participant’s role was ‘active’, i.e., it was their turn to respond to the cue and trials in which the role was ‘passive’, i.e., their task was to refrain from responding. It is reasonable to assume that different cognitive processes are at play in these types of trials and that different trajectories should result. Therefore, for further analysis we split the data into two groups: active and passive data.
First we will visually examine trajectories averaged across trials for each participant and across participants. They are averaged separately for each independent variable of interest, which would allow us to reveal different patterns depending on the variables we decide to include (if there are such patterns in the data obviously).
Our independent variables of interest are:
Now for the social conditions.
From these plots it would seem that in condition 1 only the current trial type affects movement trajectory, in condition 2 current and previous trial type affects trajectories in passive but not in active trials and in the social conditions 3 and 4 neither of the independent variables affects the shape of movement trajectory and furthermore that participants go straight for their assigned response button instead of producing curved trajectories typically found in mouse-tracking studies (similarly, however, to active trajectories in condition 2).
Instead of plotting both x,y-coordinates, we could focus our attention on just the x plane, since it is the one of bigger relevance to the task.
It appears that also examining the x plane, the only detectable difference in trajectories is the one detectable in condition 1 as influenced by the current trial type. For this reason in the remainder of this analysis we focus only on the current trial type as an independent variable of interest.
To check whether this overall pattern holds on the individual level, we can also plot average trajectories for individual participants, focusing here on the effect of current trial type, for active and passive data.
Here we can make several observations. First, even though the general pattern that we see on the group level holds for majority but not all individuals. There are 3 participants in condition 3 and 2 participants in condition 4 do exhibit trajectories that go upwards first and then to the response box (curved trajectories). For some of these individuals we can also note a slightly bigger curve in incogruent active trials but no difference in passive trials. Furthermore, all of them show shorter trajectories in passive than active trials, while most participants with straight trajectories seem to proceed all the way to the response box even when it is not their turn to respond, i.e. despite the fact that they could return to the starting position as soon as it became clear it is not their trial.
We can confirm the latter observation by also plotting a histogram of maximum y coordinate reached in passive trials. The thresholds indicated on the plot are the locations of the start boundary, the y-coordinate that had to be crossed in order for the cue to appear and the lower response box boundary. As can be seen from the plot, the majority of trajectories goes beyond that last threshold.
In order to examine the qualitative difference between individual trajectories in a more unbiased manner, we can automatically assign participants to straight vs curved motion strategy based on the shape of the majority of their trajectories. Mousetrap package provides a possibility for mapping collected trajectories to a number of trajectory prototypes frequently encountered in mouse tracking experiments, such as these:
We can map trajectories collected in our experiment to the prototypes and plot them.
We can also plot the distribution of trajectories for all the participants, focusing here on the two main types, “straight” and “curved”+“cCoM” prototypes.
For the following plots we will separate straight-trajectory participants from the curved-trajectory participants in conditions 2, 3 and 4 as they seem to behave in a qualitatively different manner. We use the prototype information to decide which participants belong to which group, i.e. if the majority of a given person’s trajectories are straight, they are assigned to the straight group and analogously for the curved trajectories.
In addition to plotting time-normalized trajectories in full, we can bin them into several intervals with the aim of further subjecting the bins to inferential testing.
Some analyses, such as looking at movement velocity profiles, require retaining trajectories in raw time. In this case, we decide how many raw time bins to create between 0 ms and some cutoff (e.g., 1500 ms) and then create a number of raw time steps. Thus, each step (i.e., coordinate pair) of a trajectory reflects the location of the mouse during some raw time bin (e.g., 500-600 ms if bins are 100 ms wide).
Once we have created such raw time bins, we can look at velocity in different types of trials. The literature suggests, for example, that stronger competition between response options should be characterized by an initial decreased velocity as competing choices inhibit each other, followed by an increase in velocity once the system converges upon a decision and the inhibition is alleviated. Thus, analyzing velocity data can allow for inferences about when commitments to a particular response are made.
In our particular case, we might also ask whether velocity is different between active and passive trials and whether participants that exhibit qualitatively different movement trajectories also differ in their velocity profiles.
Here we plot velocity profiles in binned raw time, together with approximate time in which cue appeared (mean appearance time being 294 ms).
Exploratory analysis suggests that conditions differ in how the type of trial affects movement trajectories. There seem to be notable differences in the individual Simon condition, with incongruent trials producing more curved movement path. There seem to be no consistent differences in condition 2. The social conditions 3 and 4 look most like condition 2 with some exceptions.
In this part we will carry out statistical analyses to further probe these observations. The analysis is carried out separately for 2 types of individual conditions and the social conditions.
One way to test whether trajectories differ in different conditions is by examining directly coordinates of interest. Since the x-coordinate plane is typically thought to be more relevant to the Simon task, we focus on analyzing x coordinates.
One approach is to use 101 paired-samples t tests to compare the x-coordinate of participants’ mean trajectories for two conditions at each individual time step.
The test revealed a sequence of 84, 12, 0 and 4 significant t-tests in conditions 1 to 4 respectively, on the difference between x-coordinates in congruent and incongruent trials. In order to determine what is the minimum number of significant t-tests that qualifies as a pattern, a bootstrapping procedure would be required. However, we can state already that the difference between conditions seems notable and low in conditions 2-4 by comparison with previous research (where 8 was the minimum) and with condition 1.
Other than looking at particular coordinates, we can also run tests on binned trajectories (both normalized and raw time but we will only perform the former), that we have also plotted above. For conditions 1 and 2 we perform a repeated measures 3 (bins) by 2 (trial type) ANOVA. For the social conditions we employ a mixed effects ANOVA with an additional between-subjects variable of which condition the data belongs to.
Condition 1:
| num Df | den Df | MSE | F | pes | ges | Pr(>F) | |
|---|---|---|---|---|---|---|---|
| trial.type | 1.000000 | 18.00000 | 0.0007256 | 140.27823 | 0.8862762 | 0.1778884 | 0 |
| bin | 1.267733 | 22.81919 | 0.0142317 | 267.56206 | 0.9369664 | 0.9112076 | 0 |
| trial.type:bin | 1.942834 | 34.97101 | 0.0007351 | 46.41735 | 0.7205722 | 0.1235250 | 0 |
Condition 2:
| num Df | den Df | MSE | F | pes | ges | Pr(>F) | |
|---|---|---|---|---|---|---|---|
| trial.type | 1.000000 | 18.00000 | 0.0032649 | 1.402770 | 0.0722974 | 0.0019096 | 0.2516644 |
| bin | 1.189428 | 21.40969 | 0.0370187 | 169.689498 | 0.9040969 | 0.7573580 | 0.0000000 |
| trial.type:bin | 1.158038 | 20.84469 | 0.0025087 | 1.060174 | 0.0556225 | 0.0012850 | 0.3264773 |
Social conditions:
| num Df | den Df | MSE | F | pes | ges | Pr(>F) | |
|---|---|---|---|---|---|---|---|
| condition | 1.000000 | 32.00000 | 0.0179144 | 0.0352864 | 0.0011015 | 0.0007595 | 0.8521829 |
| trial.type | 1.000000 | 32.00000 | 0.0000951 | 0.4613318 | 0.0142117 | 0.0000528 | 0.5018858 |
| condition:trial.type | 1.000000 | 32.00000 | 0.0000951 | 2.2421383 | 0.0654789 | 0.0002564 | 0.1440950 |
| bin | 1.906838 | 61.01881 | 0.0041256 | 1520.1102281 | 0.9793829 | 0.9349787 | 0.0000000 |
| condition:bin | 1.906838 | 61.01881 | 0.0041256 | 1.8053652 | 0.0534047 | 0.0167912 | 0.1746368 |
| trial.type:bin | 1.960349 | 62.73118 | 0.0000573 | 0.7298312 | 0.0222987 | 0.0000985 | 0.4834620 |
| condition:trial.type:bin | 1.960349 | 62.73118 | 0.0000573 | 0.1788361 | 0.0055576 | 0.0000241 | 0.8325036 |
The result tells us that there is an obvious significant difference in x positions in different time bins for all conditions (expected given the nature of the task). However, only in condition 1 there is a significant main effect of trial type and a significant interaction between bin and trial type.
Having dealt with coordinates, we can move on to looking at dependent measures typically examined in movement trajectories. That is, the question here is whether any of the particular measures that summarize trajectories (e.g. maximum x or y positions, area under curve, x flips) differs between congruent and incogruent trials, separately for active and passive data. As stated before, we will only examine RT and AUC here and ENT in the next subsection.
In what follows we will consider our study to employ a mixed design with 1 between-subject (condition) and 1 within-subject variables (trial type).
## [1] <NA>
## attr(,"contrasts")
## cond2vscond1 cond2vscond3 cond2vscond4
## 1 1 0 0
## 2 0 0 0
## 3 0 1 0
## 4 0 0 1
## Levels: 1 2 3 4
## Model df AIC BIC logLik Test L.Ratio p-value
## baseline 1 4 1785.322 1797.470 -888.6613
## trial.type.rt 2 5 1764.848 1780.033 -877.4239 1 vs 2 22.47472 <.0001
## condition.rt 3 8 1770.147 1794.443 -877.0736 2 vs 3 0.70065 0.8731
## rt.mix.model 4 11 1722.911 1756.318 -850.4557 3 vs 4 53.23574 <.0001
## Value Std.Error DF
## (Intercept) 736.398404 34.622346 73
## trial.typeincongruent 7.940998 5.900600 73
## conditioncond2vscond1 10.755563 48.963391 73
## conditioncond2vscond3 10.449809 47.264718 73
## conditioncond2vscond4 19.965969 50.382913 73
## trial.typeincongruent:conditioncond2vscond1 55.880342 8.344708 73
## trial.typeincongruent:conditioncond2vscond3 2.080295 8.055208 73
## trial.typeincongruent:conditioncond2vscond4 -7.805809 8.586634 73
## t-value p-value
## (Intercept) 21.2694545 5.176712e-33
## trial.typeincongruent 1.3457951 1.825335e-01
## conditioncond2vscond1 0.2196654 8.267448e-01
## conditioncond2vscond3 0.2210911 8.256387e-01
## conditioncond2vscond4 0.3962845 6.930511e-01
## trial.typeincongruent:conditioncond2vscond1 6.6965003 3.787050e-09
## trial.typeincongruent:conditioncond2vscond3 0.2582547 7.969371e-01
## trial.typeincongruent:conditioncond2vscond4 -0.9090651 3.663074e-01
Sometimes, averaging trajectories produces artifical results. For example, a smooth average trajectory for a given participant could be a result of a large number of straight trajectories that go directly to the target and discrete error type of trajectories where participant first moves directly to the wrong side and then abruptly changes direction (as one of the members of couple 7).
The main method to eliminate this possibility relies on bimodality analysis, which checks whether spatial measures are bimodally distributed.
A distribution is considered bimodal if BC > 0.555. In our case neither distribution plots nor bimodality coefficients (which are below 0.555 for all conditions) give any reason to suspect bimodal data that could blur our results.
## Model df AIC BIC logLik Test L.Ratio
## baseline 1 4 -273.4277 -261.2798 140.7138
## trial.type.auc 2 5 -289.7842 -274.5994 149.8921 1 vs 2 18.35653
## condition.auc 3 8 -361.6327 -337.3371 188.8163 2 vs 3 77.84852
## auc.mix.model 4 11 -399.9932 -366.5867 210.9966 3 vs 4 44.36049
## p-value
## baseline
## trial.type.auc <.0001
## condition.auc <.0001
## auc.mix.model <.0001
## Value Std.Error DF
## (Intercept) 0.068998206 0.02640397 73
## trial.typeincongruent 0.010795447 0.00804089 73
## conditioncond2vscond1 0.278100574 0.03734085 73
## conditioncond2vscond3 -0.032205198 0.03604539 73
## conditioncond2vscond4 -0.042267379 0.03842341 73
## trial.typeincongruent:conditioncond2vscond1 0.065250320 0.01137154 73
## trial.typeincongruent:conditioncond2vscond3 -0.006586108 0.01097703 73
## trial.typeincongruent:conditioncond2vscond4 -0.006324490 0.01170121 73
## t-value p-value
## (Intercept) 2.6131758 1.088687e-02
## trial.typeincongruent 1.3425686 1.835718e-01
## conditioncond2vscond1 7.4476235 1.528370e-10
## conditioncond2vscond3 -0.8934623 3.745458e-01
## conditioncond2vscond4 -1.1000422 2.749281e-01
## trial.typeincongruent:conditioncond2vscond1 5.7380392 2.038578e-07
## trial.typeincongruent:conditioncond2vscond3 -0.5999902 5.503708e-01
## trial.typeincongruent:conditioncond2vscond4 -0.5404986 5.904985e-01
Since we have seen that participants tend to employ two qualitatively different movement patterns, a question we can ask is whether two groups also lead to qualitatively different statistical results. To that end we run t-tests on the individual level. Our prediction is that curved participants but not straight participants show significantly higher RT and AUC in incongruent trials compared to congruent trials.
| personid | prototype | measure | t | df | p | lower.conf | upper.conf | estimate | effect |
|---|---|---|---|---|---|---|---|---|---|
| c2p1 | curved | RT | 1.395 | 155 | 0.91748 | -Inf | 43.872 | 20.067 | 0.111 |
| c2p1 | curved | AUC | 0.477 | 155 | 0.68295 | -Inf | 0.022 | 0.005 | 0.038 |
| c2p13 | curved | RT | -1.944 | 151 | 0.02685 | -Inf | -10.602 | -71.224 | 0.156 |
| c2p13 | curved | AUC | -23.739 | 151 | 0 | -Inf | -0.185 | -0.199 | 0.888 |
| c2p19 | curved | RT | -2.647 | 157 | 0.00447 | -Inf | -21.485 | -57.306 | 0.207 |
| c2p19 | curved | AUC | -4.309 | 157 | 1e-05 | -Inf | -0.014 | -0.023 | 0.325 |
| c2p2 | curved | RT | -2.012 | 156 | 0.02298 | -Inf | -5.593 | -31.501 | 0.159 |
| c2p2 | curved | AUC | -1.905 | 156 | 0.02933 | -Inf | -0.001 | -0.01 | 0.151 |
| c2p6 | curved | RT | -0.583 | 156 | 0.28022 | -Inf | 13.983 | -7.616 | 0.047 |
| c2p6 | curved | AUC | -1.447 | 156 | 0.07497 | -Inf | 0.001 | -0.008 | 0.115 |
| c2p10 | straight | RT | -1.299 | 158 | 0.09798 | -Inf | 6.67 | -24.337 | 0.103 |
| c2p10 | straight | AUC | -1.659 | 158 | 0.04956 | -Inf | 0 | -0.013 | 0.131 |
| c2p12 | straight | RT | -0.561 | 158 | 0.28778 | -Inf | 19.719 | -10.118 | 0.045 |
| c2p12 | straight | AUC | -0.341 | 158 | 0.36662 | -Inf | 0.007 | -0.002 | 0.027 |
| c2p14 | straight | RT | 0.964 | 156 | 0.83181 | -Inf | 32.717 | 12.046 | 0.077 |
| c2p14 | straight | AUC | -0.67 | 156 | 0.25192 | -Inf | 0.007 | -0.005 | 0.054 |
| c2p15 | straight | RT | 1.735 | 156 | 0.95762 | -Inf | 74.399 | 38.078 | 0.138 |
| c2p15 | straight | AUC | -0.171 | 156 | 0.4323 | -Inf | 0.012 | -0.001 | 0.014 |
| c2p16 | straight | RT | 0.05 | 158 | 0.51998 | -Inf | 34.835 | 1.026 | 0.004 |
| c2p16 | straight | AUC | 0.081 | 158 | 0.53207 | -Inf | 0.041 | 0.002 | 0.006 |
| c2p17 | straight | RT | -0.947 | 157 | 0.17261 | -Inf | 14.042 | -18.781 | 0.075 |
| c2p17 | straight | AUC | -0.783 | 157 | 0.21741 | -Inf | 0.009 | -0.008 | 0.062 |
| c2p18 | straight | RT | 0.143 | 151 | 0.55667 | -Inf | 42.324 | 3.361 | 0.012 |
| c2p18 | straight | AUC | 2.131 | 151 | 0.98263 | -Inf | 0.044 | 0.025 | 0.171 |
| c2p20 | straight | RT | -1.223 | 158 | 0.11149 | -Inf | 5.998 | -17.022 | 0.097 |
| c2p20 | straight | AUC | 0.4 | 158 | 0.6551 | -Inf | 0.026 | 0.005 | 0.032 |
| c2p3 | straight | RT | 0.58 | 158 | 0.71872 | -Inf | 33.437 | 8.682 | 0.046 |
| c2p3 | straight | AUC | 1.225 | 158 | 0.88876 | -Inf | 0.02 | 0.009 | 0.097 |
| c2p4 | straight | RT | -0.093 | 158 | 0.46287 | -Inf | 26.292 | -1.572 | 0.007 |
| c2p4 | straight | AUC | -1.113 | 158 | 0.1336 | -Inf | 0.003 | -0.006 | 0.088 |
| c2p5 | straight | RT | 0.356 | 156 | 0.63891 | -Inf | 39.086 | 6.924 | 0.029 |
| c2p5 | straight | AUC | 1.156 | 156 | 0.87536 | -Inf | 0.025 | 0.01 | 0.092 |
| c2p7 | straight | RT | 0.606 | 157 | 0.72741 | -Inf | 40.275 | 10.801 | 0.048 |
| c2p7 | straight | AUC | 1.956 | 157 | 0.97386 | -Inf | 0.025 | 0.014 | 0.154 |
| c2p8 | straight | RT | 0.022 | 155 | 0.5086 | -Inf | 37.869 | 0.488 | 0.002 |
| c2p8 | straight | AUC | -0.055 | 155 | 0.47811 | -Inf | 0.009 | 0 | 0.004 |
| c2p9 | straight | RT | -0.835 | 156 | 0.20263 | -Inf | 13.54 | -13.777 | 0.067 |
| c2p9 | straight | AUC | 0.301 | 156 | 0.61797 | -Inf | 0.019 | 0.003 | 0.024 |
| personid | prototype | measure | t | df | p | lower.conf | upper.conf | estimate | effect |
|---|---|---|---|---|---|---|---|---|---|
| c3p13 | curved | RT | -5.88 | 143 | 0 | -Inf | -90.836 | -126.433 | 0.441 |
| c3p13 | curved | AUC | -2.022 | 143 | 0.02252 | -Inf | -0.01 | -0.053 | 0.167 |
| c3p14 | curved | RT | -0.773 | 146 | 0.2203 | -Inf | 20.242 | -17.746 | 0.064 |
| c3p14 | curved | AUC | -2.179 | 146 | 0.01546 | -Inf | -0.007 | -0.027 | 0.177 |
| c3p7 | curved | RT | -1.856 | 153 | 0.03272 | -Inf | -3.618 | -33.455 | 0.148 |
| c3p7 | curved | AUC | -1.447 | 153 | 0.07503 | -Inf | 0.002 | -0.012 | 0.116 |
| c4p1 | curved | RT | 1.041 | 147 | 0.85028 | -Inf | 46.406 | 17.92 | 0.086 |
| c4p1 | curved | AUC | -0.381 | 147 | 0.35195 | -Inf | 0.011 | -0.003 | 0.031 |
| c4p9 | curved | RT | 0.046 | 154 | 0.51825 | -Inf | 39.039 | 1.052 | 0.004 |
| c4p9 | curved | AUC | -1.531 | 154 | 0.06389 | -Inf | 0.002 | -0.026 | 0.122 |
| c3p1 | straight | RT | -0.499 | 146 | 0.30924 | -Inf | 21.07 | -9.095 | 0.041 |
| c3p1 | straight | AUC | -0.781 | 146 | 0.21813 | -Inf | 0.01 | -0.009 | 0.064 |
| c3p10 | straight | RT | -0.422 | 154 | 0.33673 | -Inf | 30.704 | -10.518 | 0.034 |
| c3p10 | straight | AUC | 1.329 | 154 | 0.90711 | -Inf | 0.018 | 0.008 | 0.106 |
| c3p11 | straight | RT | -0.285 | 152 | 0.38812 | -Inf | 19.83 | -4.12 | 0.023 |
| c3p11 | straight | AUC | 0.75 | 152 | 0.77283 | -Inf | 0.014 | 0.004 | 0.061 |
| c3p12 | straight | RT | -1.882 | 156 | 0.03083 | -Inf | -3.85 | -31.847 | 0.149 |
| c3p12 | straight | AUC | -0.056 | 156 | 0.4776 | -Inf | 0.013 | 0 | 0.005 |
| c3p15 | straight | RT | 0.672 | 150 | 0.74857 | -Inf | 46.125 | 13.314 | 0.055 |
| c3p15 | straight | AUC | -0.569 | 150 | 0.28523 | -Inf | 0.01 | -0.005 | 0.046 |
| c3p16 | straight | RT | 0.17 | 146 | 0.56729 | -Inf | 27.089 | 2.52 | 0.014 |
| c3p16 | straight | AUC | 2.03 | 146 | 0.97792 | -Inf | 0.041 | 0.023 | 0.166 |
| c3p17 | straight | RT | -0.239 | 151 | 0.40558 | -Inf | 25.801 | -4.362 | 0.019 |
| c3p17 | straight | AUC | 0.167 | 151 | 0.5661 | -Inf | 0.027 | 0.003 | 0.014 |
| c3p18 | straight | RT | 0.271 | 146 | 0.60652 | -Inf | 42.604 | 5.989 | 0.022 |
| c3p18 | straight | AUC | -1.229 | 146 | 0.11052 | -Inf | 0.004 | -0.011 | 0.101 |
| c3p19 | straight | RT | -0.104 | 154 | 0.4588 | -Inf | 29.382 | -1.963 | 0.008 |
| c3p19 | straight | AUC | -1.328 | 154 | 0.09314 | -Inf | 0.003 | -0.011 | 0.106 |
| c3p2 | straight | RT | -0.966 | 141 | 0.16783 | -Inf | 16.666 | -23.346 | 0.081 |
| c3p2 | straight | AUC | 0.718 | 141 | 0.76299 | -Inf | 0.017 | 0.005 | 0.06 |
| c3p20 | straight | RT | 0.449 | 154 | 0.67291 | -Inf | 47.658 | 10.168 | 0.036 |
| c3p20 | straight | AUC | -1.244 | 154 | 0.10768 | -Inf | 0.002 | -0.006 | 0.1 |
| c3p21 | straight | RT | 0.93 | 157 | 0.82311 | -Inf | 53.914 | 19.4 | 0.074 |
| c3p21 | straight | AUC | 0.528 | 157 | 0.70082 | -Inf | 0.024 | 0.006 | 0.042 |
| c3p22 | straight | RT | 0.044 | 157 | 0.51759 | -Inf | 34.365 | 0.894 | 0.004 |
| c3p22 | straight | AUC | -1.092 | 157 | 0.13835 | -Inf | 0.002 | -0.005 | 0.087 |
| c3p3 | straight | RT | -0.796 | 154 | 0.21359 | -Inf | 17.85 | -16.55 | 0.064 |
| c3p3 | straight | AUC | -0.12 | 154 | 0.45246 | -Inf | 0.013 | -0.001 | 0.01 |
| c3p4 | straight | RT | -1.643 | 151 | 0.05122 | -Inf | 0.255 | -35.111 | 0.133 |
| c3p4 | straight | AUC | -1.399 | 151 | 0.08189 | -Inf | 0.002 | -0.011 | 0.113 |
| c3p5 | straight | RT | 1.039 | 148 | 0.84967 | -Inf | 43.702 | 16.85 | 0.085 |
| c3p5 | straight | AUC | 0.795 | 148 | 0.78611 | -Inf | 0.018 | 0.006 | 0.065 |
| c3p6 | straight | RT | -0.007 | 148 | 0.49712 | -Inf | 38.788 | -0.17 | 0.001 |
| c3p6 | straight | AUC | 0.827 | 148 | 0.79516 | -Inf | 0.013 | 0.004 | 0.068 |
| c3p8 | straight | RT | 0.801 | 151 | 0.78784 | -Inf | 55.674 | 18.159 | 0.065 |
| c3p8 | straight | AUC | -0.011 | 151 | 0.49554 | -Inf | 0.011 | 0 | 0.001 |
| c3p9 | straight | RT | 0.729 | 153 | 0.76634 | -Inf | 46.665 | 14.266 | 0.059 |
| c3p9 | straight | AUC | -0.028 | 153 | 0.48878 | -Inf | 0.017 | 0 | 0.002 |
| c4p10 | straight | RT | -1.809 | 152 | 0.03623 | -Inf | -2.194 | -25.789 | 0.145 |
| c4p10 | straight | AUC | 0.294 | 152 | 0.61527 | -Inf | 0.009 | 0.001 | 0.024 |
| c4p11 | straight | RT | -0.112 | 156 | 0.45545 | -Inf | 33.352 | -2.423 | 0.009 |
| c4p11 | straight | AUC | -2.586 | 156 | 0.00531 | -Inf | -0.009 | -0.024 | 0.203 |
| c4p12 | straight | RT | 1.671 | 153 | 0.95165 | -Inf | 63.491 | 31.902 | 0.134 |
| c4p12 | straight | AUC | 0.844 | 153 | 0.79993 | -Inf | 0.013 | 0.004 | 0.068 |
| c4p14 | straight | RT | -0.915 | 147 | 0.18071 | -Inf | 19.881 | -24.604 | 0.075 |
| c4p14 | straight | AUC | -0.473 | 147 | 0.31859 | -Inf | 0.005 | -0.002 | 0.039 |
| c4p15 | straight | RT | -0.341 | 157 | 0.36675 | -Inf | 21.726 | -5.642 | 0.027 |
| c4p15 | straight | AUC | -0.182 | 157 | 0.4281 | -Inf | 0.011 | -0.001 | 0.014 |
| c4p16 | straight | RT | -0.896 | 150 | 0.18579 | -Inf | 18.103 | -21.379 | 0.073 |
| c4p16 | straight | AUC | -0.348 | 150 | 0.36402 | -Inf | 0.01 | -0.003 | 0.028 |
| c4p17 | straight | RT | -0.296 | 146 | 0.3839 | -Inf | 27.868 | -6.064 | 0.024 |
| c4p17 | straight | AUC | -0.841 | 146 | 0.20079 | -Inf | 0.007 | -0.007 | 0.069 |
| c4p18 | straight | RT | -0.568 | 150 | 0.2856 | -Inf | 20.002 | -10.438 | 0.046 |
| c4p18 | straight | AUC | 0.359 | 150 | 0.63979 | -Inf | 0.017 | 0.003 | 0.029 |
| c4p2 | straight | RT | 1.413 | 152 | 0.92019 | -Inf | 67.399 | 31.045 | 0.114 |
| c4p2 | straight | AUC | 0.691 | 152 | 0.75461 | -Inf | 0.02 | 0.006 | 0.056 |
| c4p20 | straight | RT | 0.678 | 148 | 0.75049 | -Inf | 51.983 | 15.101 | 0.056 |
| c4p20 | straight | AUC | -1.238 | 148 | 0.10881 | -Inf | 0.003 | -0.008 | 0.101 |
| c4p3 | straight | RT | 0.428 | 150 | 0.66536 | -Inf | 48.702 | 10.006 | 0.035 |
| c4p3 | straight | AUC | -1.553 | 150 | 0.06129 | -Inf | 0.001 | -0.013 | 0.126 |
| c4p5 | straight | RT | 0.372 | 151 | 0.64466 | -Inf | 27.297 | 5.006 | 0.03 |
| c4p5 | straight | AUC | -1.356 | 151 | 0.08855 | -Inf | 0.002 | -0.01 | 0.11 |
| c4p6 | straight | RT | -0.718 | 151 | 0.23707 | -Inf | 23.264 | -17.807 | 0.058 |
| c4p6 | straight | AUC | -0.182 | 151 | 0.42773 | -Inf | 0.009 | -0.001 | 0.015 |
| c4p7 | straight | RT | 0.367 | 136 | 0.64283 | -Inf | 42.443 | 7.696 | 0.031 |
| c4p7 | straight | AUC | 0.292 | 136 | 0.61457 | -Inf | 0.02 | 0.003 | 0.025 |
| c4p8 | straight | RT | -0.026 | 132 | 0.48956 | -Inf | 35.559 | -0.572 | 0.002 |
| c4p8 | straight | AUC | 0.229 | 132 | 0.59041 | -Inf | 0.015 | 0.002 | 0.02 |
Mouse trajectories are typically thought to result from a number of cognitive processes that depend on different factors within a trial. In order to examine the contribution of these processes to the overall trajectory, two main methods have been proposed: Principal Components Analysis (PCA) and visualization of beta weights.
To conduct PCA we use time-normalized trajectories averaged within each participant. We will focus on components underlying x coordinates.
Given the resulting plots, we can see that in condition 1 the component that distinguishes between trial types is the third one. In condition 2, the second component has reverse shape in active and passive trials, while third seems to distinguish trial types. In condition 3, only component 3 is markedly different in incongruent active trials. In condition 4, component 2 has reverse shape in active and passive trials, while component 3 is different mostly in passive congruent trials.
Following Scherbaum et al., we also examine trajectory angles. (TODO) They have applied the following steps to visualize different trajectory components:
# plot trials split by 4 predictors
# plot beta weights against time
Finally, given that we are working with continuous data, we can try to apply analyses that come from dynamical systems approach to cognition to see if such measures can detect regularities that are missed by traditional statistical methods.
In some cases, it may be helpful to measure the complexity of mouse trajectories. For example, if both response alternatives simultaneously attract participants’ mouse movement (relative to only one), this additional stress might manifest as less smooth, more complex, and fluctuating trajectories. Some mouse-tracking studies have used “x-flips”, others opted for sample entropy, a measure of predictability of trajectory given a number of surrounding coordinates. We have seen above that a measure of x-flips did not deliver statistically significant results. Here we calculate sample entropy, using default settings.
##
## Paired t-test
##
## data: sample_entropy by trial.type
## t = -9.0891, df = 18, p-value = 3.802e-08
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -0.03273935 -0.02044578
## sample estimates:
## mean of the differences
## -0.02659257
In condition 1 we see a highly significant difference in sample entropy between congruent and incongruent trials, indicating that participants’ movement in incongruent trials was more erratic.
| num Df | den Df | MSE | F | pes | ges | Pr(>F) | |
|---|---|---|---|---|---|---|---|
| trial.type | 1 | 16 | 0.0000157 | 2.367721 | 0.1289066 | 0.0005076 | 0.1434087 |
| role.type | 1 | 16 | 0.0021265 | 1.448251 | 0.0830027 | 0.0403382 | 0.2463130 |
| trial.type:role.type | 1 | 16 | 0.0000328 | 1.239965 | 0.0719239 | 0.0005544 | 0.2819273 |
In condition 2, there seems to be no effect on movement entropy by either role or trial type.
Another question one might ask in a joint action scenario is whether participants’ responses are more correlated within an experimental pair than across different pairs and whether this depends on the availability of visual information about the co-actor’s movements.
The measure that is most frequently used in the DST community is cross-recurrence quantification analysis (CRQA) that is an index of the coupling between two time series. Crudely put, it relies on reconstructing phase spaces of the systems from a given data and checking whether the states that the systems visit are close to each other.
In order to carry out such analysis, 3 hyperparameters need to be set or determined:
This can be done either based on the literature or sampled from and optimized for a number of trials in the data and then applied to the remaining trials. Given that we are aware of no studies that have applied CRQA to mouse-tracking data, we opt for the latter option. We estimate the parameters based on a random sample of trials and compute CRQA measures for all trials for all pairs.
Having computed the CRQA metrics, one can visualize the different trials, as well as perform statistical analyses on the measures obtained.
RQA plots visualize how the states of the system evolve over time, while the measures we can look at are as follows:
| Measure | Definition |
|---|---|
| recurrence rate | how often the system visits the same state |
| determinism | how often the same sequences repeat |
| meanline and maxline | how long are repeating sequences |
| entropy | how many repeating patterns are there |
| laminarity and trapping time | how long the time series remains in the same state |
| trend | whether it’s stationary |
The most widely used of these are recurrence rate (RR) and determinism.
Just to get a feel for these measures we can first examine the example trials in which coupling (as indexed by RR and determinism) has been estimated to be low and high.
Subsequently we can see if coupling is higher within couples than across and whether it is affected by the presence of visual information.
Starting with the latter, the question is whether coupling among couples in condition 3 differs from coupling among couples in condition 4.
##
## Welch Two Sample t-test
##
## data: RR by condition
## t = -1.2325, df = 12.056, p-value = 0.2413
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
## -3.1740196 0.8797203
## sample estimates:
## mean in group 3 mean in group 4
## 8.497376 9.644526
From the plots it appears that only recurrence rate seems higher in condition 4 but the difference is not significant.
Judging whether coupling within couples is indeed present typically implies a comparison between coupling calculated for real couples and coupling calculated for so-called surrogate (virtual, fake) couples. That is, we take data from the same people but form new pairs from them and compute the same CRQA measures. Of course, given that people perform the same task, some amount of coupling is expected to hold just by virtue of the task requirements. However, if there is significant coupling between people who perform the task together, it should be higher between people who actually performed it together than between people who merely performed the same task.
Given no difference between conditions 3 and 4 we will focus here only on data from condition 4 and for simplicity examine only the recurrence rate measure.
## comparison.group p.value
## 1 fake0 0.45821
## 2 fake1 0.90668
## 3 fake2 0.96468
## 4 fake3 0.46011
## 5 fake4 0.96139
## 6 fake5 0.84992
## 7 fake6 0.96506
## 8 fake7 0.79204
## 9 fake8 0.18629
## 10 fake9 0.68983
From the plot as well as a t-test it seems that real couples do not show higher level of coupling than fake pairs.
This, together with a lack of social Simon effect in the standard trajectory measures, as well as trajectories showing spatial division of labor, suggests that people performed the task individually, rather than approaching it as a social, joint activity.
Fractal measures detrmine how variability scales with sample size. Low level of self-similarity typically indicates a random process, medium level - self-organization, while high level an influence of external constraints on the unfolding of cognitive dynamics. (TODO)
Social conditions
In the social conditions it seems that both role and trial type have an significant effect on movement complexity.
We can observe all these results on the following combined plots.
It appears that movement complexity is the highest in the condition in which participants had to carry out the whole task by themselves and lowest in the individual go-nogo condition (in active trials). There does not seem to be a difference depending on whether participants see each other’s cursors (conditions 3 and 4). In addition, in the social conditions incongruent trials have led to higher movement complexity, movement in active trials was more complex than in passive trials but there was no interaction between these factors.